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ABSTRACT 

This  paper  arseases  the  predictix-e  ability  of  th^  Box-Jenkins 
methodology  when  utilized  in  an  ongoing  setting.   Three  procedures 
are  utilized  to  update  the  original  forecasts  generated  from  the 
Box-Jenkins  models:  adaptive  forecasting,  reestimation  and  re- 
identification.   The  results  indicate  that  constant  monitoring 
of  the  structure  and  parameters  of  the  time  series  models  are 
necessary  through  time.   It  appears  that  adaptive  forecasting 
techniques  are  insufficient  in  performing  this  function. 


A  COMPARATIVE  ANALYSIS  OF  THE  PREDICTIVE  ABILITY  OF  ADAPTIVE 

FORECASTING,  REESTIMATION  AND  REIDENTIFICATION  USING 

BOX-JENKINS  TIME  SERIES  ANALYSIS 


INTRODUCTION 

A  considerable  amount  of  evidence  has  been  reported 
recently  concerning  the  applicability  of  Box-Jenkins  (here- 
after referred  to  as  BJ)  time  series  analysis  in  predictive 
settings. [1} [3] [4] [5] [6] [7 J     These  studies  have  assessed  the 
predictive  ability  of  BJ  models  in  various  manners: 
(1)  simple  presentation  of  forecast  accuracy  without  signi- 
ficance testing  [lj[5jf    (2)  comparative  analysis  of  the 
forecasting  accuracy  of  BJ  time  series  models  relative  to 
certain  simplistic  models  [7],    (3)  comparative  analysis  of 
the  forecasting  accuracy  of  the  BJ  time  series  models  and 
econometric  models  [6]^   and  (4)  comparative  analysis  of  the 
accuraqy  oi  BJ  models  and  management  forecast.:,.  [3]  [4] 

The  results  of  the  above  studies  have  consistently 
demonstrated  the  po^^erful  nature  of  the  autoregressive  inte- 
grated moving  average  (hereafter  referred  to  as  ARIMA) 
models.   These  models  are  particularly  attractive  consider- 
ing the  possibility  of  describing  highly  variant  time  series 
behavior.  Moreover,  the  BJ  methodology  utilizes  an  inherent 
structure  in  the  determination  of  the  functional  form  of  the 
model  eliminating  potential  arbitrariness  in  the  model  selec- 
tion process. 
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An  important  issue  which  has  not  been  systematically 
researched  concerns  the  predictive  ability  of  the  BJ  ARIMA 
models  in  an  on-going  setting.  As  more  new  observations  of 
a  time  series  become  available,  what  procedures  should  be 
utilized  in  updating  either  the  identified  time  series  model 
or  its  resultant  forecasts?^ 

In  this  paper,  we  examine  specifically  three  possible 
techniques  which  may  be  employed  to  update  BJ  predictions  as 
more  data  become  available:   adaptive  forecasting,  reestima- 
tion  and  respecification.   The  time  series  data  bases  which 
we  examined  are  comprised  of  quarterly  earnings  data  from  a 
sample  of  30  New  York  Stock  Exchange  firms. 

ARIMA  MODELS 
The  class  of  linear  models  most  appropriate  for 
quarterly  earnings  data  is  the  ARIMA  model  adjusted  for 
seasonality  factors,  commonly  referred  to  as  the  general 
multiplicative  seasonal  model.   Let  the  difference  operator 
V  be  defined  by: 

VZ,  =  (1-B)  Z^  »  2,  -  Z^.^  ^) 


where  B  is  called  a  backshift  operator  in  the  general  form: 
®     ^t  "  ^t-n-     "^^^  entire  general  multiplicative  seasonal 
ARIMA  model  can  be  represented  as  follows: 

1-H.  Ill  "^  flJ^"'!^''''^  ^ith  the  detection  of  radical  departures  in 
the  nature  of  the  tzwe  series  data  under  examination.     Detection  of 
such  turning  points  oay  be  undertaken  utilizing  the  parabolic  mask  pro- 
cedures suggested  by  Brown  [2] .  i^^uu^xxw  mctsx  pro 
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*p  (B)  *p  (b5)vV  Z^  =  e^  (B)  0Q  (B^)  a^  +  e^     (2) 

where: 

Z^  =  the  value  of  the  original  series  at  time  t. 

Z^  =  (Z^  -  u)  when  d  =  D  =  0. 

a^  =  the  residuals  or  "white  noise"  at  time  t. 

4,    (B)    =   1   -    (j)jB <t>pBP 

$    (B^)    =   1   -    «^B* *pB^^ 

7^  '    (1-B^)^ 
s 

eq(B)  =  1  -  e^B e^B^ 

0q(b5)    =   1   -   ©^B^    - GqB^^ 

e_  =  deterministic  trend  parameter. 

Equation  (2)  is  referred  to  as  the  ARIMA  pdq  X  PDQ 
model.  •  The  designations  p  and  q  refer  to  the  order  of  the 
autoregressive  and  moving  average  polynomials  (1>(B)  and  9(B). 
The  letters  d  and  D  designate  the  degree  of  consecutive  and 
seasonal  differencing  necessary  to  attain  stationarity.  The 
letters  P  and  Q  refer  to  the  order  of  the  seasonal  autoregres- 
sive  and  moving  average  polynomials  *(B  )  and  0(B). 

UPDATING  PROCEDURES 

Assuming  that  we  have  a  pdq  X  PDQ  model  for  a  given 
time  series  (Z,,  Z.,  ♦•••  Z  ),  what  updating  techniques  can 


be  implemented  when  observations  (Z  ^j,  Z  ,»  *•*'  ^n+m^ 
become  available?   In  other  words,  what  adjustments  to  the 
original  moiel  or  the  predictions  o£  the  original  model  are 
necessary  to  take  into  consideration  the  information 
impounded  in  the  new  observations?  Three  possible  alterna- 
tives are  explored  in  this  paper:   adaptive  forecasting, 
reestimation  and  reidentification. 

In  the  utilization  of  adaptive  forecasting,  the  func- 
tional form  of  the  original  time  series  model  remains 
unchanged.  However,  the  remaining  predictions,  in  *the  fore- 
cast horizon  are  modified  to  incorporate  the  informational 
content  of  the  new  data.   For  example,  assume  data  base 
(Zj^,  Z2,  ••••»  Z  )  is  utilized  to  predict  Z^^,  and  Z^^2* 
Then, 

Z^(l)  =   forecasted  value  of  Z.^,  at  time  t 
Z^(2)  =  forecasted  value  of  Z.^-  *^  time  t. 

As  a  new  observation  of  data  becomes  available  (Z^^,) , 
adaptive  forecasting  may  be  utilized  to  update  the  remaining 
forecasts,  Z^(2)  in  this  case.   The  change  in  the  original 
forecast  of  Z^^2  "^^V  ^^  represented  as  follows: 

Z^+id)  -  ZJ2)  =  factor  •  (Z^^j  -  ZJl)  )         (3) 

The  LHS  of  equation  (3)  is  the  difference  between  the 
updated  forecast  of  Z^^2  *"<*  ^^®  original  forecast  of  Z^^2* 
This  difference  is  due  to  the  informational  content  of  the 


additional  data.   The  RHS  of  equation  (3)  reveals  the 
specific  nature  of  the  updating  process,  a  model  specific 
factor  multiplied  by  the  forecast  error  associated  with  the 
prediction  of  Z.^,. 

In  utilizing  reestimation  procedures  the  identification 
stage  of  the  BJ  iterative  model  building  process  is  bypassed 
and  the  parameters  of  the  original  model  are  simply  reesti- 
mated.  The  new  data  are  appended  to  the  existing  data  base 
and  rerun  through  the  non- linear  least  squares  algorithm. 
For  example,  if  the  original  model  were  an  autoregressive 
process  of  order  one  with  a  <|>,  parameter  equal  to  .6,  the 
functional  form  of  the  model  would  remain  the  same  but  the 
(^,  parameter  might  be  changed  dependent  upon  the  impact  of 
the  new  data.   This  technique  differs  from  adaptive  fore- 
casting in  that  parameter  values  of  the  original  model  may 
be  changed.   It  is  similar  to  adaptive  forecasting  in  that 
the  structure  of  the  model  remains  unchanged. 

Finally,  the  third  alternative,  reidentification, 
involves  a  complete  reapplication  of  the  BJ  methodology  when- 
ever a  new  observation  of  data  becomes  available.   The  ad- 
justed data  base  is  input  sequentially  through  the  iterative 
stages  of  identification,  estimation  and  diagnostic  checking. 
This  technique  differs  from  the  first  two  alternatives  by 
allowing  for  structural  change  in  the  original  model.   For 
example,  if  the  original  model  were  an  autoregressive  process 
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Nelson  [6,  pp.  157-159]  provides  several  exanqples  of  how  the  factor 

term  is  derived.   In  actuality,  the  factor  term  is  dependent  upon  the 

parameters  of  the  original  model. 


of  order  one,  the  reidentification  technique  would  ignore 
this  information  and  allow  for  the  possibility  of  any  ARIMA 
model  to  be  identified. 

Thus,  adaptive  forecasting  is  clearly  the  most  conven- 
ient mechanism  available  for  updating  BJ  forecasts.   Reesti- 
mation  is  somewhat  more  complex  considering  that  parameter 
values  must  be  re-examined.  Reidentification  is  the  most 
involved  updating  technique  examined;  it  employs  all  three 
modelling  stages  in  the  BJ  methodology  with  the  addition  of 
each  new  data  point. 

DATA 

A  sample  of  New  York  Stock  Exchange  firms  was  obtained 
by  utilizing  the  following  sampling  criteria: 

1.  Only  calendar  year  reporting  firms  were  selected. 

2.  Quarterly  earnings  were  reported  in  the  Wall  Street 
Journal  Index  from  1959  to  1974. 

3,.   kandoin  sampling  techniques  were  employed  subject  to 
the  above  criteria  until  a  sample  size  of  30  was 
obtained.   (See  Table  1  for  a  listing  of  the  sample 
firms.) 

The  first  sampling  criterion  was  motivated  by  pragmatic 
considerations.   By  focusing  attention  on  calendar  year  firms, 
data  acquisition  was  concentrated  upon  specific  time  intervals 
surrounding  the  interim  reporting  dates.   Since  data  procure- 
ment was  performed  manually,  only  specific  sections  of  each 
yearly  volume  of  the  Wall  Street  Journal  Index  needed  to  be 
referenced. 


Table  1 

LIST  OF  SAMPLE  FIRMS 

1.  American  Home  Products  Corporation 

2.  Anchor  Hocking  Glass  Corporation 

3.  Armco  Steel  Corporation 

4.  Bausch  and  Lomb  Optical  Corporation 

5.  Baxter  Laboratories  Incorporated 

6.  Belden  Corporation 

7.  Beneficial  Finance  Corporation 

8.  Campbell  Red  Lake  Mines  Ltd. 

9.  Coca-Cola  Corporation 

10.  Crane  Company 

11.  Diebold  Incorporated 

12.  Duquesne  Light  Company 

13.  Federal-Mogul  Bower  Bearings 

14.  Gardner -Denver  Company  ' 

15.  Goodyear  Tire  §  Rubber  Company 

16.  Hercules  Incorporated 

17.  Interstate  Power  Company 

18.  Koppers  Incorporated 

19.  Liggett  §  Myers  Tobacco 

20.  Long  Island  Lighting  Company 

21.  National  Starch  §  Chemical  Corporation 

22.  Niagra  Mohawk  Power  Company 

23.  Pfizer  Incorporated 

24.  Pullman  Incorporated 

25.  Rohm  §  Haas  Company 

26.  St.  Regis  Paper  Company 

27.  Texas  Utilities  Company 

28.  Timken  Company 

29.  Union  Carbide  Corporation 
1.0.  Wm.  Wrigley  Jr.  Company 

The  second  criterion  was  necessary  to  fulfill  the  sug- 
gested minimum  data  requirements  of  the  BJ  methodology.  [1, 
p.  18]      Although  the  results  of  this  study  may  not  be  gener- 
alized to  all  New  York  Stock  Exchange  firms  or  other  types  of 
data,  we  feel  that  the  quarterly  earnings  data  are  representa- 
tive of  the  types  of  data  previously  examined  in  conjunction 
with  the  BJ  methodology. 


METHODOLOGY 

Data  bases  were  compiled  for  each  of  the  30  sample 
firms  consisting  of  64  quarterly  earnings  numbers.   They  en- 
compassed the  time  period  from  1953  to  1974.   However,  the 
last  three  years  (12  quarters)  were  deleted  initially  from 
the  model  identification  process.   These  actual  quarterly 
earnings  numbers  were  held  back  to  assess  the  accuracy  of  the 
BJ  predictions.  Thus,  for  each  sample  firm,  a  52  observation 
data  base  (quarterly  earnings  from  1959  to  1971)  w^s  utilized 
to  identify  the  most  appropriate  BJ  model.  After  identifica- 
tion of  the  models  and  the  estimation  of  their  parameters, 
they  were  then  utilized  to  predict  the  next  12  quarterly 
earnings  numbers  for  each  firm  (quarterly  earnings  from  1972- 
1974).^ 

Since  our  primary  concern  was  the  predictive  ability  of 
BJ  models  as  new  observations  of  data  became  available,  fore- 
casts of  the  remaining  quarterl-*'  earnings  numbers  in  the 
forecast  horizon  were  updated  each  time  we  appended  a  new 
observation  to  the  data  base.   Specifically,  when  we  appended 
the  actual  first  quarter  earnings  numbers  for  1972  to  the 
existing  52  observation  data  bases,  we  were  interested  in  the 
impact  these  had  upon  the  predictions  of  the  11  remaining 
quarterly  earnings  numbers  for  each  sample  firm.   This  process 
was  repeated  an  additional  10  times  until  the  only  quarterly 


We  selected  12  gimrters  or  three  years  as  the  forecast  horizon  to 
minimize  short  run  aberrations  in  a  particular  year  which  could  poten- 
tially bias  the  results  of  the  study. 


earnings  number  remaining  in  the  forecast  horizon  was  the 
fourth  quarter  earnings  of  1974. 

Although  an  examination  of  the  time  series  properties 
of  quarterly  earnings  data  is  beyond  the  scope  of  this  paper, 
the  impact  that  reestimation  and  reidentification  had  upon  the 
structure  of  the  originally  identified  BJ  model  is  of  interest, 
We  observe  that  the  first  updating  technique  examined  in  this 
paper,  adaptive  forecasting,  does  not  alter  the  structure  of 
the  originally  identified  BJ  model.   Hence,  no  model  informa- 
tion  is  reported  for  this  technique. 


Table  2 

THE  EFFECT  OF  REESTIMATION  ON  THE  TIME 
SERIES  MODELS  OF  GARDNER- DENVER  CO. 


NOB 

pdq  X  PDQ 

^0 

^1 

^2 

^3 

^7 

52 

004  X  010 

244.80 

-.56 

-.58 

-.45 

.90 

53 

251.37 

-.55 

-.53 

-.42 

.90 

54 

251.99 

-.56 

-.54 

-.41 

.90 

55 

243.68 

-.57 

-.54 

-.40 

.88 

56 

246.64 

-.56 

-.53 

-.40 

.88 

57 

239.12 

-.56 

-.54 

-.41 

.88 

58 

230.15 

-.57 

-.55 

-.41 

.88 

59 

264.50 

-.55 

-.59 

-.50 

.92 

60 

186.91 

-.62 

-.65 

-.51 

.85 

61 

299.49 

-.50 

-.56 

-.50 

.97 

62 

152.45 

-.80 

-.96 

-.59 

.47 

63 

181.75 

-.34 

-.82 

-.10 

.44 
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Table  3 

THE  EFFECT  OF  REIDENTIFICATIQN  ON  THE  TIKffi 
SERIES  MODELS  OF  GARDNER -DENVER  CO. 


NOB 

pdq 

X  PDQ 

*1 

^0 

^1 

^2 

'z 

^7 

^1 

^2 

52 

004 

X  010 

244.80 

-.56 

-.58 

-.45 

.90 

53 

102 

X  Oil 

.62 

122,60 

-.13 

=  51 

54 

004 

X  010 

250.59 

-.56 

-.54 

-.41 

.90 

55 

11 

242.86 

-.57 

-.54 

-.40 

.88 

56 

tt 

246.57 

-.56 

-.53 

-.40 

.88 

57 

tt 

239.19 

-.56 

-.54 

-.41 

.88 

58 

«t 

229.15 

-.57 

-.55 

-.41 

.87 

59 

101 

X  Oil 

.64 

136.10 

.55 

60 

006 

X  012 

340.73 

-.65 

-.46 

-.34 

.79 

.36 

.39 

61 

005 

X  Oil 

328.22 

-.57 

-.35 

-.27 

.83 

.52 

62 

101 

X  Oil 

.63 

108.15 

.63 

63 

002 

X  010 

309.54 

-.56 

.62 

where:     NOB  =  the  number  of  observations  in  the  data  base. 


The  Gardner-Denver  Co.  time  series  models  were 
selected  as  indicative  of  the  types  of  changes  reestimation 
and  reidentification  had  upon  the  original  time  series 
models.   The  intention  o£  this  example  is  to  provide  an 
intuitive  feeling  as  to  the  cifferential  effects  of  the  two 
metho'ds.   Tables  2  and  3  depict  the  changes  in  the  original 
model  of  Gardner- Denver  Co.  lAen  reestimation  and  reidenti- 
fication procedures  were  undertaken.   The  original  BJ  model 
identified  for  Gardner- Denver  Co.  using  a  52  observation 
data  base  can  be  represented  in  pdq  X  PDQ  notation  as 
004  X  010.   There  are  four  moving  average  parameters  at  lags 
one,  two,  three  and  seven.   Seasonal  differencing  was  util- 
ized to  attain  stationarity  and  a  deterministic  trend  para- 
meter was  detected. 
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IVhen  the  55rd  observation  was  appended  to  the  data 
base,  reestimation  procedures  required  that  the  identifica- 
tion stage  of  the  BJ  methodology  be  bypassed.   The  004  X  010 
model  structure  was  assumed  to  be  still  appropriate  and  the 
existing  parameters  were  simply  updated  by  inputing  the 
expanded  data  base  into  the  estimation  stage  of  the  BJ 
methodology.   The  parameters  of  the  004  X  010  model  appear 
very  stable  when  the  number  of  observations  ranges  from 
52-58.   However,  the  parameters  oscillate  markedly  between 
59-63  observations;  perhaps  casting  doubt  on  the  propriety 
of  the  004  X  010  model  for  these  expanded  data  bases. 

Since  reidentification  procedures  avoid  any  a-priori 
assumptions  regarding  the  propriety  of  the  original  BJ  model, 
the  functional  form  of  the  most  appropriate  model  (see  pdq  X 
PDQ  column  in  Table  3)  may  change.   This  procedure  resulted 
in  a  different  BJ  model  for  S3  observations:   102  X  Oil  with 
a  return  to  the  original  structure  between  observations 
54-58.   This  pattern  is  disrupted  between  observations  59-63, 
Thus,  in  addition  to  parameter  changes  the   functional  form 
of  the  model  changes  as  well. 

HYPOTHESES 

Although  Box  and  Jenkins  (i,  pp,  134-35]    and  Nelson 
[6,  pp.  157-59]   discuss  the  use  of  adaptive  forecasting  to  up- 
date predictions,  no  comparative  assessment  is  available  con- 
cerning the  use  of  more  involved  updating  procedures,  i.e., 
reestimation  and  reidentification.   In  order  to  provide 


12 

information  regarding  the  comparative  predictive  ability  of 
these  three  updating  techniques,  the  following  hypotheses 
were  tested: 

H  :1  As  more  data  become  available,  the  updating  of 
°    BJ  predictiorxs  using  reestimation  results  in 
more  accurate  predictions  than  adaptive  fore- 
casting. 

H  :2  As  more  data  become  available,  the  updating  of 
°    BJ  predictions  using  reidentification  results  in 
more  accurate  predictions  than  reestimation. 

RESULTS 

The  accuracy  of  the  predictions  generated  from  adaptive 
forecasting,  reestimation  and  reidentification  was  assessed  by 
utilizing  Theil's  U  Coefficient,  /s;   The  U  Coefficient  is 
confined  to  the  interval  between  zero  and  one.   That  is,  the 
closer  the  value  of  the  U  Coefficient  is  to  zero  the  more 
accurate  is  the  forecast,  and  the  farther  removed  from  zero 
the  less  accurate.   The  U  Coefficient  is  computed  in  the  fol- 
lowing manner: 


nI^cv^ 


)^ 


U  = 


(4) 


M 


)     ?   \^ 1     2 
n    1    "*  n    1 


where: 

p.  =  predictions  of  quarterly  earnings 

A-  *  actual  quarterly  earnings 

n  =  the  number  of  predictions 
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The  evaluations  made  in  this  paper  were  based  on  paired 
comparisons  between  the  sample  means  of  the  U  Coefficients. 
The  U  Coefcicients  were  calcul ited  by  reference  to  the  pre- 
dictions generated  from  the  BJ  models  using  the  three  alter- 
native techniques.   Sample  means  were  compiled  by  computing 
the  average  value  of  the  U  Coefficients  across  all  30  firms. 
These  values  are  reported  in  Table  4. 

Table  4 
SA14PLE  MEANS  OF  U  COEFFICIENTS 


NOB 

P 

53 

11 

54 

10 

55 

9 

56 

8 

57 

7 

58 

6 

59 

S 

60 

4 

61 

3 

62 

2 

63 

1 

Adaptive    Reestimation 


Re identification 


238 
244 
247 
258 
260 
265 
279 
284 
319 
323 
421 


.228 
.224 
.215 

.226 
.216 
.198 
.209 

.205 
.237 
.250 


.230 
.215 
.214 
.219 
.201 
.180 
.196 
.197 
.209 
.202 
.334 


where:     NOB  ~  number  of  observations  in  data  base 

P  =  nuisber  of  quarter ly  earnings  predictions  generated 


For  purposes   of  testing   the   statistical   significance 
of  H^:!   and  H    :2,    the  one- tailed  "student's  matched-pair" 
t-test  was  utilised.      Table   5   sunmarizes   the   results  of   the 
statistical   analysis. 
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Table  5 

SIGNIFICANCE  TESTING 


NOB 

P 

%--^ 

«o--2 

53 

11 

.10 

NS 

54 

10 

.05 

NS 

55 

9 

.05 

NS 

56 

8 

.05 

.10 

57 

7 

.05 

.05 

58 

6 

.01 

.05 

59 

5 

.01 

.10 

60 

4 

.01 

.10 

61 

3 

.05 

.05 

62 

2 

.05 

.05 

63 

1 

.05 

NS 

where:     NOB  =  number  of  observations  in  data  base 
P   =  number  of  quarterly  earnings  predic- 
tions generated 
SS  ~  non-significant  difference 


Table  5  is  interpreted  in  the  following  manner:   I£ 
there  is  a  significant  difference  in  the  updating  techniques 
an  a  level  between  .01-. 10  is  indicated.   If  there  is  a  non- 
significant difference  in  the  predictions,  NS  is  indicated. 
Specifically,  when  NOB  =  53  predictions  are  generated  for  11 
quarterly  earnings  numbers  using  three  different  updating 
techniques.   The  eleven  forecasts  generated  by  reestimation 
were  significantly  superior  (a=.10)  to  the  eleven  forecasts 
generated  by  adaptive  forecasting  (K  :1).  However,  there  was 
no  significant  difference  (NS)  between  the  forecasts  generated 
by  reestimation  and  reidentification  (H  : 2) . 
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INTERPRETATION  OF  RESULTS  AND  CONCLUSIONS 

The  results  reported  herein  indicate  that  adaptive 
forecasting  provided  significantly  larger  forecast  errors 
when  compared  to  reestimation.   This  finding  is  consistently 
demonstrated  regardless  of  the  number  of  predictions  in  the 
forecast  horizon.   The  comparisons  between  reestimation  and 
reidentification  are  not  as  straightforward.   When  the  number 
of  predictions  in  the  forecast  horizon  varied  from  2-8,  rees- 
timation provided  significantly  larger  forecast  errors.   When 
the  number  of  predictions  ranged  between  9-11  and  1  there  was 
no  significant  difference. 

Of  course,  these  results  are  subject  to  the  sampling 
procedures  employed,  the  specific  data  examined  and  the  error 
measure  selected.  However,  given  the  recent  proliferation  of 
BJ  applications  in  the  literature,  more  information  is  neces- 
sary regarding  the  adaptive  ability  of  the  BJ  models.   The 
results  suggest  that  constant  ^nonitoring  of  the  adequacy  of 
the  structure  of  the  BJ  model  as  well  as  its  parameters  is 
necessary  to  maintain  the  predictive  ability  of  the  model. 
If  appears  that  adaptive  forecasting  may  be  insufficient  to 
provide  this  function. 
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